Simulations of estimating causal effects using undersmoothed HAL
Seraphina Shi
2023-05-10
Simulation 1
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-5 + W + 2.25A -0.5WA)]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.838778 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.684755 1st Qu.:0.6223 1st Qu.:0.0000
## Median : 0.006970 Median :2.0148 Median :0.0000
## Mean : 0.006617 Mean :2.1300 Mean :0.4688
## 3rd Qu.: 0.715963 3rd Qu.:3.4240 3rd Qu.:1.0000
## Max. : 4.256679 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.410 2.888 2.749 4.187 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.9237 2.2579 2.3098 3.6095 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.4189 1.7457 1.9354 3.1514 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.197 1.557 2.651 5.000
n = 200
## The average lambda of CV-HAL: 0.0142 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0044 (= 0.3129 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0118, 0.0115, 0.0115, 0.0116, 0.0118, 0.0121, 0.0123, 0.0124, 0.0124, 0.0124, 0.0124 (= [0.8446, 0.8299, 0.8286, 0.829, 0.8406, 0.8601, 0.8737, 0.8797, 0.8794, 0.8814, 0.8846] * lambda_CV## The average fitting time for CV-HAL: 4.9150 seconds## The average fitting time for globally undersmoothed HAL: 7.3811 seconds## The average fitting time for locally undersmoothed HAL: 31.2348 seconds
n = 500
## The average lambda of CV-HAL: 0.0056 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0017 (= 0.2954 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0041, 0.004, 0.004, 0.004, 0.004, 0.0041, 0.0042, 0.0043, 0.0044, 0.0044, 0.0044 (= [0.7114, 0.6953, 0.6941, 0.6953, 0.6956, 0.7073, 0.7306, 0.7531, 0.7661, 0.7696, 0.7751] * lambda_CV## The average fitting time for CV-HAL: 5.3269 seconds## The average fitting time for globally undersmoothed HAL: 10.3145 seconds## The average fitting time for locally undersmoothed HAL: 57.3556 seconds
n = 1000
## The average lambda of CV-HAL: 0.0029 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0008 (= 0.2740 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0018, 0.0017, 0.0016, 0.0016, 0.0016, 0.0017, 0.0018, 0.0019, 0.0019, 0.0019, 0.002 (= [0.6072, 0.5747, 0.567, 0.56, 0.5592, 0.5821, 0.6146, 0.6403, 0.6668, 0.6747, 0.6785] * lambda_CV## The average fitting time for CV-HAL: 5.8674 seconds## The average fitting time for globally undersmoothed HAL: 11.6748 seconds## The average fitting time for locally undersmoothed HAL: 63.8782 seconds
Simulation 2
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 + 2W + 5sin(A^{1.5}) + 2WA)]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-4.023488 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.683524 1st Qu.:0.5998 1st Qu.:0.0000
## Median : 0.002623 Median :2.0083 Median :0.0000
## Mean :-0.002187 Mean :2.1171 Mean :0.0665
## 3rd Qu.: 0.666584 3rd Qu.:3.3949 3rd Qu.:0.0000
## Max. : 3.819893 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.408 2.743 2.703 4.078 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.8347 2.2646 2.3032 3.6241 5.0000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.4634 1.7649 1.9340 3.1372 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.292 1.531 2.534 5.000
n = 200
## The average lambda of CV-HAL: 0.0036 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0016 (= 0.3371 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0036, 0.0035, 0.0034, 0.0034, 0.0034, 0.0034, 0.0034, 0.0034, 0.0035, 0.0034, 0.0034 (= [1, 0.9924, 0.9842, 0.9853, 0.9863, 0.9869, 0.9887, 0.9887, 0.9897, 0.9824, 0.9841] * lambda_CV## The average fitting time for CV-HAL: 4.9046 seconds## The average fitting time for globally undersmoothed HAL: 6.5215 seconds## The average fitting time for locally undersmoothed HAL: 19.8505 seconds
## [1] "proportion of simulations that cannot compute empirical SD:"## 1.2 1.1 1 0.6952 0.4833 0.336 0.2336 0.1624 0.1129 0.0785 0.0546
## 0.740 0.748 0.752 0.774 0.830 0.844 0.838 0.864 0.898 0.902 0.924
## 0.0379 0.0264 0.0183 0.0127 0.0089 0.0062 0.0043 0.003 0.0021 0.0014 0.001
## 0.938 0.950 0.954 0.944 0.958 0.954 0.960 0.952 0.956 0.962 0.966Simulation 3
Data structure: \(O = (W, A, Y)\)
- U - exogenous variables
- W - baseline covariate that is a measure of body condition
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- \(U_Y \sim Uniform(min = 0, max = 1)\)
- Structural equations F and endogenous variables:
- \(W = U_W\)
- \(A = bound(2 - 0.5W + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 - 3W + 4A + \mathbf{I}(A>2) * 5sin((0.8A)^2 - 2.6) )]\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W A Y
## Min. :-3.829350 Min. :0.0000 Min. :0.0000
## 1st Qu.:-0.684888 1st Qu.:0.5926 1st Qu.:0.0000
## Median : 0.000602 Median :2.0179 Median :0.0000
## Mean :-0.001718 Mean :2.1279 Mean :0.4387
## 3rd Qu.: 0.678175 3rd Qu.:3.4197 3rd Qu.:1.0000
## Max. : 3.818122 Max. :5.0000 Max. :1.0000## Summary of A given W < -1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 1.424 2.819 2.744 4.195 5.000## Summary of A given -1 < W <= 0:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.911 2.260 2.311 3.645 5.000## Summary of A given 0 < W <= 1:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.0000 0.3843 1.8105 1.9431 3.1824 5.0000## Summary of A given 1 < W:## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 1.248 1.519 2.555 5.000
n = 200
## The average lambda of CV-HAL: 0.0026 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0009 (= 0.2837 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.0024, 0.0024, 0.0024, 0.0024, 0.0025, 0.0025, 0.0025, 0.0024, 0.0025, 0.0025, 0.0026 (= [0.8706, 0.8682, 0.8726, 0.8927, 0.9219, 0.9584, 0.9378, 0.9093, 0.9588, 0.9761, 0.9854] * lambda_CV## The average fitting time for CV-HAL: 3.7293 seconds## The average fitting time for globally undersmoothed HAL: 6.0426 seconds## The average fitting time for locally undersmoothed HAL: 24.0037 seconds
n = 500
## The average lambda of CV-HAL: 0.0009 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0003 (= 0.2800 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 8e-04, 8e-04, 8e-04, 8e-04, 8e-04, 9e-04, 8e-04, 8e-04, 9e-04, 9e-04, 9e-04 (= [0.9227, 0.9227, 0.9237, 0.9206, 0.9183, 0.9397, 0.9147, 0.9201, 0.9661, 0.9833, 0.9895] * lambda_CV## The average fitting time for CV-HAL: 6.1899 seconds## The average fitting time for globally undersmoothed HAL: 14.7461 seconds## The average fitting time for locally undersmoothed HAL: 85.6112 seconds
n = 1000
## The average lambda of CV-HAL: 0.0003 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0001 (= 0.2699 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04, 3e-04 (= [0.949, 0.949, 0.949, 0.9451, 0.9413, 0.9233, 0.9045, 0.9306, 0.9681, 0.9648, 0.9764] * lambda_CV## The average fitting time for CV-HAL: 11.0246 seconds## The average fitting time for globally undersmoothed HAL: 23.4035 seconds## The average fitting time for locally undersmoothed HAL: 118.1803 seconds
Simulation 4
Data structure: \(O = (W_1, W_2, W_3, W_4, W_5, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- Structural equations F and endogenous variables:
- \(W \sim N(\mu_W, \Sigma_W)\)
- \(A = bound(0.1W_1 + 0.2W_2 + 0.5W_3 + 0.15W_4 - 0.05W_5 - 0.01W_3W_5 + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-7 -W_1 +
2W_2 - 0.5W_4 - W_1W_3 + 4A + 0.5AW_2)]\)
- where \(\mu_W = \begin{bmatrix}0 \\0 \\5 \\1 \\1 \\\end{bmatrix}\), and \(\Sigma_W = \begin{bmatrix}1&0.8&1.35&0.1&0.03 \\0.8&1&1.2&0.15&0.03 \\1.35&1.2&2.25&0.075&0.225 \\0.1&0.15&0.075&0.25&0.1125 \\0.03&0.03&0.225&0.1125&0.09 \\\end{bmatrix}\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W1 W2 W3 W4
## Min. :-3.35759 Min. :-4.18291 Min. :-0.5103 Min. :-0.9502
## 1st Qu.:-0.64759 1st Qu.:-0.66601 1st Qu.: 4.0031 1st Qu.: 0.6678
## Median : 0.02145 Median : 0.02657 Median : 5.0508 Median : 0.9990
## Mean : 0.02423 Mean : 0.02460 Mean : 5.0447 Mean : 1.0043
## 3rd Qu.: 0.70289 3rd Qu.: 0.71364 3rd Qu.: 6.0807 3rd Qu.: 1.3422
## Max. : 4.15076 Max. : 3.74670 Max. :10.9968 Max. : 2.7898
## W5 A Y
## Min. :-0.2897 Min. :0.000 Min. :0.0000
## 1st Qu.: 0.7766 1st Qu.:1.619 1st Qu.:0.0000
## Median : 1.0038 Median :2.596 Median :1.0000
## Mean : 1.0064 Mean :2.580 Mean :0.6199
## 3rd Qu.: 1.2370 3rd Qu.:3.554 3rd Qu.:1.0000
## Max. : 2.4618 Max. :5.000 Max. :1.0000
Simulation 5
Data structure: \(O = (W_1, W_2, W_3, W_4, W_5, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 5
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Exogenous variables:
- \(U_A \sim Normal(\mu=0, \sigma^2 = 1^2)\)
- \(U_A \sim Normal(\mu=0, \sigma^2 = 2^2)\)
- Structural equations F and endogenous variables:
- \(W \sim N(\mu_W, \Sigma_W)\)
- \(A = bound(0.1W_1 + 0.2W_2 + 0.5W_3 + 0.15W_4 - 0.05W_5 - 0.01W_3W_5 + U_A, min=0, max=5)\)
- \(Y = \mathbf{I}[U_Y < expit(-10 -W_1 +
2W_2 - 0.5W_4 - 0.5W_1W_3 + 4A + \mathbf{I}(A>2) *5sin((0.8A)^2 -
2.6))]\)
- where \(\mu_W = \begin{bmatrix}0 \\0 \\5 \\1 \\1 \\\end{bmatrix}\), and \(\Sigma_W = \begin{bmatrix}1&0.8&1.35&0.1&0.03 \\0.8&1&1.2&0.15&0.03 \\1.35&1.2&2.25&0.075&0.225 \\0.1&0.15&0.075&0.25&0.1125 \\0.03&0.03&0.225&0.1125&0.09 \\\end{bmatrix}\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in [0,5]\), the causal dose-response curve
## W1 W2 W3 W4
## Min. :-4.10305 Min. :-3.992762 Min. :-0.723 Min. :-1.0645
## 1st Qu.:-0.66856 1st Qu.:-0.665766 1st Qu.: 3.993 1st Qu.: 0.6574
## Median : 0.01409 Median : 0.004221 Median : 5.007 Median : 0.9974
## Mean : 0.01345 Mean : 0.004542 Mean : 5.008 Mean : 0.9924
## 3rd Qu.: 0.68434 3rd Qu.: 0.676313 3rd Qu.: 6.014 3rd Qu.: 1.3343
## Max. : 3.37515 Max. : 4.074948 Max. :10.222 Max. : 2.8006
## W5 A Y
## Min. :-0.1632 Min. :0.000 Min. :0.000
## 1st Qu.: 0.7565 1st Qu.:1.604 1st Qu.:0.000
## Median : 0.9909 Median :2.561 Median :0.000
## Mean : 0.9929 Mean :2.554 Mean :0.476
## 3rd Qu.: 1.2283 3rd Qu.:3.509 3rd Qu.:1.000
## Max. : 2.2176 Max. :5.000 Max. :1.000
Simulation 6
Data structure: \(O = (W_1, W_2, W_3, A, Y)\)
- W - baseline covariates
- A - treatment level based on W, continuous between 0 and 1
- Y - outcome, indicator of an event
Underlying data generating process, \(P_{U,X}\)
- Structural equations F and endogenous variables:
- \(W_1 \sim Uniform(o,1)\)
- \(W_2 \sim Bernoulli(\mu=0, \sigma^2 = 2^2)\)
- \(W_3 \sim N(W_1, 0.25*exp(2W_1))\)
- \(A \sim Beta(v(W)\mu(W), v(W)[1-\mu(W)])\)
- \(Y \sim Bernoulli(Q_0(A,W))\)
- where:
- \(v(W) = exp(1 + 2W_1expit(W3))\)
- \(\mu(W) = expit(0.03 - 0.8log(1+W_2) + 0.9exp(W_1)W_2 - 0.4arctan(W_3+2)W_2W_1)\)
- \(\bar{Q}_0(A,W) = expit(-2 + 1.5A + 5A^3 - 2.5W_1 + 0.5AW_2 - log(A)W_1W_2 + 0.5A^{3/4}W_1W_3)\)
Outcome of interest: \(E_0[Y|a,W]\), \(a \in (0,1])\), the causal dose-response curve
## W1 W2 W3 A
## Min. :0.0000212 Min. :0.0000 Min. :-3.20646 Min. :0.008172
## 1st Qu.:0.2550912 1st Qu.:0.0000 1st Qu.: 0.07964 1st Qu.:0.483313
## Median :0.5016742 Median :1.0000 Median : 0.80953 Median :0.661615
## Mean :0.5026247 Mean :0.6984 Mean : 0.80890 Mean :0.632873
## 3rd Qu.:0.7530768 3rd Qu.:1.0000 3rd Qu.: 1.53093 3rd Qu.:0.807451
## Max. :0.9999523 Max. :1.0000 Max. : 5.10017 Max. :0.999333
## Y
## Min. :0.000
## 1st Qu.:0.000
## Median :0.000
## Mean :0.483
## 3rd Qu.:1.000
## Max. :1.000
n = 200
## The average lambda of CV-HAL: 0.0090 (= 1 * lambda_CV )## The average lambda of globally undersmoothed HAL: 0.0023 (= 0.2632 * lambda_CV )## The average lambdatime of locally undersmoothed HAL: 0.006, 0.0058, 0.0057, 0.0057, 0.0056, 0.0056, 0.0056, 0.0054, 0.0053, 0.0051, 0.0049, 0.0044, 0.0043, 0.0043, 0.0042, 0.0045, 0.0045 (= [0.6913, 0.676, 0.6703, 0.666, 0.6611, 0.6609, 0.6546, 0.6404, 0.6239, 0.603, 0.5776, 0.5095, 0.4942, 0.4902, 0.4755, 0.5031, 0.5107] * lambda_CV## The average fitting time for CV-HAL: 40.4222 seconds## The average fitting time for globally undersmoothed HAL: 46.4000 seconds## The average fitting time for locally undersmoothed HAL: 159.3079 seconds
